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Training  and  Simulation 


FOREWORD 


The  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences  (ARI) ,  working  in  cooperation  with  the  U.S.  Army 
Training  and  Doctrine  Command  (TRADOC)  and  with  TRADOC' s  schools, 
conducts  research  for  development  on  ways  to  achieve  more  cost- 
effective  training  for  the  Army.  In  1987  ARI  joined  with  TRADOC 
and  the  U.S.  Army  Ordnance  Center  and  School  (USAOC&S)  to  form  a 
partnership  at  the  Aberdeen  Proving  Ground  in  Maryland  dedicated 
to  identifying  and  solving  maintenance  training  problems.  The 
partnership  was  defined  by  a  memorandum  of  understanding  entitled 
"Establishment  of  a  Training  Technology  Field  Area  (TTFA)  at  the 
USAOC&S"  and  dated  10  May  1987. 

The  present  research  report  is  one  result  of  that  partner¬ 
ship.  The  work  was  carried  out  by  members  of  the  Automated 
Instructional  Systems  Technical  Area  (formerly  the  Logistics 
Training  Technologies  Technical  Area)  of  ARI's  Training  Research 
Laboratory  to  supply  practical  guidelines  for  designing  mainte¬ 
nance  training  that  will  result  in  a  high  degree  of  skill  reten¬ 
tion  and  transfer. 

This  and  other  products  of  the  TTFA  were  briefed  to  the 
Deputy  Assistant  Commandant,  USAOC&S,  in  October  1988.  It  will 
be  used  to  help  develop  model  training  instruction  and  plan  addi¬ 
tional  TTFA  projects  at  USAOC&S.  Its  value  was  recognized  by  MG 
Ball,  Commandant,  USAOC&S,  in  a  letter  of  appreciation  to  the 
authors . 


EDGAR  M.  JOHNSON 
Technical  Director 
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APPLICATION  OF  TRAINING  RESEARCH  LITERATURE  TO  MAINTENANCE 
PERFORMANCE  TRAINING 


EXECUTIVE  SUMMARY 


Requirement: 

Generate  training  development  guidelines  to  improve  skill 
acquisition,  retention,  and  transfer  for  wheel  vehicle  mainte¬ 
nance  personnel.  Identify  research  needed  to  complete  or  vali 
date  such  guidelines. 


Method: 

We  reviewed  data  on  maintenance  performance  deficiency  and 
current  training  practices  at  the  U.S.  Army  Ordnance  Center  and 
School  (USAOC&S)  and  in  U.S.  Army  Forces  Command  (FORSCOM)  units. 
This  review  showed  that  we  need  to  better  train  and  job  aid  main¬ 
tenance  personnel.  We  then  analyzed  the  literature  on  skill  ac¬ 
quisition,  retention,  and  transfer.  From  this  literature  we 
derived  training  and  research  implications.  Finally  we  generated 
a  set  of  10  guidelines  for  effective  training. 


Findings: 


1.  Failure  among  wheel  vehicle  mechanics  to  perform  to 
standard  is  a  significant  but  predictable  problem  in  the  Army. 

2.  Many  valid  techniques  and  strategies  exist,  which  could 
reduce  this  condition  to  a  negligible  level. 

3.  These  techniques  and  strategies  are  not  being  used  at 
APG,  or  are  being  used  ineffectively. 

4.  To  implement  them  correctly,  wheel  vehicle  maintenance 
trained  needs  to  shift  away  from  a  "stand-up  lecture/content 
oriented"  philosophy  to  one  that  is  "workshop/ task  oriented." 

5.  The  techniques  and  strategies  are  fully  consistent  with 
and  support  TRADOC's  systems  approach  to  training  (SAT).  They 
are  validly  implemented  by  the  Harless  Workshop  on  Front-End 
Analysis,  Job  Aiding,  and  Course  Development. 
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6.  The  techniques  and  strategies  have  been  condensed  into  a 
checklist  for  improving  63W  training. 


Use  of  Findings: 

1.  The  findings  can  be  used  to  design  a  model  program  of 
instruction  at  the  U.S.  Army  Ordnance  Center  &  School  as  well  as 
at  other  TRADOC  schools. 

2.  Section  4.2,  "Training  issues  not  well  resolved,"  will 
be  of  particular  interest  to  TTFA's  since  it  identifies  unre¬ 
solved  training  issues. 

3.  We  present  our  "Checklist  for  Effective  Training  Devel¬ 
opment"  below.  The  checklist  outlines  our  findings  and  guidance 
in  a  simple  format.  Use  the  checklist  to  check  developed  train¬ 
ing  and  to  guide  the  development  of  training.  Use  the  report, 
and  especially  Chapter  5,  to  aid  you  in  understanding,  applying, 
and  explaining  the  checklist. 


Checklist  for  Effective  Training  Development 


Prologue: 

1.  Use  of  the  checklist  will  reduce  net  training  costs  and 
increase  unit  readiness  by  increasing  skill  retention  and  trans¬ 
fer  to  new  equipment. 

2.  Even  if  a  few  checklist  items  are  applied  or  the  check¬ 
list  is  applied  less  stringently,  significant  improvements  will 
result. 


3.  Since  many  of  the  checklist  items  may  demand  extra  re¬ 
sources,  they  will  have  to  be  applied  to  a  prioritized  list  of 
tasks.  For  high  priority  tasks,  all  the  items  can  be  applied 
"full  strength."  For  lower  priority  tasks,  some  of  the  items  can 
be  implemented. 
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Checklist  for  Effective  Training  Development 


1.  Identify  problem  tasks. 

2.  Identify  problem  steps  in  task:  Discriminations, 

Generalizations , 

Easily  forgotten  steps, 
'•Instinctive*'  performance. 


3.  Integrate  B  K  &  S  with  task  training. 

4.  Use  a  workshop  training  procedure. 

5.  Use  5-P  principle:  Prime 

Prompt 

Perform 

Practice  isolated  task 
Practice  task  with  related  tasks. 


6.  Raise  the  standards  and  use  remedial  training 
to  achieve  them. 

7.  Sequence  the  training:  Overview  to  specific, 

Familiar  to  unfamiliar. 

Concrete  to  abstract. 

8.  Doing  part  task  training: 

o  Break  long  tasks  into  groups  of  steps; 
o  Use  natural  groupings  and  give  them  meaningful  labels; 

o  Teach  each  group  as  a  separate  training  exercise; 

o  'Train  students  to  recognize  completion  of  each  step; 
o  Provide  extra  practice  for  difficult  steps; 
o  Highlight  abstract  materials  and  difficult  steps; 
o  Decrease  help  as  students  gain  skill  on  a  task; 

9.  Train  for  retention  and  ability  to  work  new  equipment. 

10.  Improve  job  aiding  and  Technical  Manual  training. 
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APPLICATION  OP  TRAINING  RESEARCH  LITERATURE 
TO  MAINTENANCE  PERFORMANCE  TRAINING 

1 . 0  INTRODUCTION 


1 . 1  Overview 

This  report  is  one  of  a  series  in  the  Training  Technology  Field 
Activity  (TTFA)  program  to  improve  automotive  maintenance  training  at 
the  U.S.  Army  Ordnance  Center  &  School  (USAOCS) .  In  the  report  we 
review  maintenance  performance  problems,  current  training  practices,  and 
the  training  research  literature  over  the  last  25  years.  On  the  basis 
of  that  literature,  we  recommend  ways  to  improve  introductory  training 
of  Wheel  Vehicle  Mechanics  (63W10s) .  In  so  doing,  we  underscore  a 
number  of  fundamental  discrepancies  between  current  training  practice 
and  well  established,  scientifically  based  principles  of  effective 
instruction. 

1 . 2  Background 

a.  The  Army  Research  Institute  joined  USAOCS  and  TRADOC's  Training 
Technology  Activity  (TTA)  in  a  TTFA  at  Aberdeen,  Maryland.  The  goal  of 
the  TTFA  was  to  identify  shortfalls  in  maintenance  performance  and  then 
build  one  or  more  model  course  segments  to  demonstrate  how  those  short¬ 
falls  could  be  reduced  through  improved  training  methodology.  To 
identify  the  shortfalls,  TTFA  staff  audited  the  63W10  Course  at 
Aberdeen,  and  collected  end-of -block  test  data.  We  then  assessed 
performance  in  FORSCOM  and  reserve  component  maintenance  units.  These 
activities  were  designed  to  point  to  tasks  not  being  performed 
satisfactorily.  But,  they  would  not  tell  us  how  to  revise  the  training 
to  reduce  the  performance  deficiencies. 

b.  The  project  summarized  by  this  report  was  designed  to  help  solve 
that  problem  by  deriving  research-based  guidelines  on  how  to  make  63W10 
training  more  effective.  Our  strategy  was  to  develop  a  set  of  general 
principles  that  are  not  task  specific  and  then  apply  them  to  the 
deficient  tasks.  To  better  focus  our  efforts  and  economize  our 
resources,  we  chose  to  start  with  an  existing,  candidate  set  of 
guidelines  developed  by  Harless  (1982)  and  to  use  the  research  to 
validate  those  guidelines  and/or  revise  them  in  accordance  with 
conclusions  from  the  research. 

c.  We  chose  Harless  (Appendix  1)  because  TRADOC  has  sponsored 
workshops  in  this  methodology  at  10  of  its  schools,  and  because  the 
methodology  is  completely  consistent  with  TRADOC  Regulation  350-7  on  the 
systems  approach  to  training  development.  Harless  provides  a  set  of  job 
aids  which  show  you  how  to  implement  each  of  the  stages  of  training 
development  outlined  in  the  regulation.  His  two-week  workshop  teaches 
how  to  use  the  job  aids  to  analyze  performance  problems  and  produce  job 
aids  and  course  material  where  skill  deficiency  is  the  problem. 
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2 . 0  METHOD 


2 . 1  Overview 

a.  Review  of  Performance  Problems  and  Current  Training  Practice.  We 
began  our  analysis  by  examining  reports  and  data  on  performance 
deficiencies  among  wheel  vehicle  maintenance  trainees  and  journeymen 
(Kern  &  Hayes,  1983;  Macpherson  et  al.,  1988a;  Mirabella,  1988;  Ramsay 
et  al.,  1988a,  1988b).  We  also  reviewed  current  training  practice  and 
philosophy  in  the  63W10  POI  at  USAOCS  (Ramsay  et  al.,  1988a,  1988b). 

The  reason  for  these  reviews  was  to  establish  the  need  to  produce 
guidelines  for  improved  training  at  APG. 

b.  Research  Literature  Analysis.  We  designed  a  data  analysis  form 
with  three  columns.  Column  1  is  for  conclusions  derived  from  reviews 
of  training  research  literature  and  literature  reviews.  Column  2  was 
designed  for  comments  on  what  the  literature  means  for  instructional 
technology,  in  particular  the  Harless  methods.  Column  3  outlines 
additional  research  needed  to  fill  gaps  in  knowledge  about  effective 
training  methods. 

c.  Development  of  Guidelines.  Here  we  used  an  approach  from  a 
companion  TTFA  project  designed  to  produce  rules  for  developing 
classroom  visual  aids  (Ramsay  et  al.  1988b).  There  we  wrote  rules, 
illustrated  how  current  practices  violate  the  rules  and  then  showed  how 
to  use  them  properly.  We  used  the  same  approach  to  write  guidelines 
for  improving  63W10  training,  i.e.  from  the  data  base  of  the  literature 
analysis,  we  extracted  a  set  of  rules.  In  a  separate  report 
(Macpherson,  Patterson,  and  Mirabella,  1988b)  we  will  show  how  current 
practices  might  be  improved  by  applying  the  rules. 


2.2  Maintenance  Performance  Deficiencies  and  Current  Training 
Practice 

a.  We  reviewed  and  summarized  results  from  other  TTFA  studies  as 
well  as  previous  Army  research.  Ramsay  et  al.  (1988a)  describe,  in  great 
detail,  how  the  63W10  program  is  conducted  at  APG  and  provide  some  data 
on  end-of-annex  (EOF)  failure  rates  (i.e.,  per  cent  students  failing  to 
achieve  a  score  of  70%  on  a  written  test) .  The  EOA  data  for  Phase  1 
(Basic  Skills  &  Knowledge)  are  failure  rates  for  16  classes  for  which 
data  were  currently  available.  In  this  report  we  cite  average  failure 
rates  per  annex.  Test  data  for  Phase  2  (MOS-Specif ic  Training) ,  came 
from  8  classes  for  which  examination  scores  were  available  at  the  time 
of  the  study.  We  cite  data  for  1  written  EOA  test  and  for  the  written 
portion  of  the  end-of-course  test  since  these  had  notable  failure 
rates.  (Other  tests  had  zero  or  near  zero  failure  rates.) 

b.  Ramsay  et  al.  (1988a)  also  summarize  a  study  by  TRADOC's 
Analysis  Command  (TRAC) .  TRAC  measured  hands-on  performance  for  5  tasks 
at  5  FORSCOM  locations.  The  criterion  for  passing  was  stringent. 
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Soldiers  (63Ws  and  63Bs)  had  to  pass  every  step  with  no  intervention  by 
a  supervisor  in  order  to  pass  the  test.  We  cite  over  all  results  in  this 
report. 

c.  A  study  by  Macpherson  et  al.  (1988)  applied  the  ARI  Skill 
Retention  Model  to  a  number  of  wheel  vehicle  maintenance  tasks  of 
special  concern  to  SMEs  at  APG  or  actually  observed  in  FORSCOM  units  as 
part  of  the  TTFA's  front-end  analysis  of  maintenance  performance.  We 
identify  3  tasks  subject  to  severe  skill  decay  following  last  practice. 

d.  Finally,  we  cite  representative  failure  rates  from  a  study  by 
Kern  and  Hayes  (1983) ,  in  which  hands-on  performance  by  236  wheel 
vehicle  mechanics  was  observed. 


2*3  Research  Literature  Analysis 

a.  To  focus  our  analysis  of  the  research  literature  on  skill 
training,  we  emphasized  applied  military  research,  especially  the  work 
of  ARI  in  its  skill  retention  program  conducted  from  about  1979  to  about 
1985.  We  were,  however,  led  to  a  very  large  body  of  basic  research  on 
memory,  which,  while  neither  military  nor  applied,  was  very  useful.  It 
verified  much  of  what  was  in  Harless,  but  also  pointed  to  some  major 
gaps  in  the  Harless  methodology. 

b.  ARI 1 s  skill  retention  work  began  with  empirical  studies  in  a 
variety  of  TRADOC  schools,  including  USAOCS.  It  climaxed  in  a  highly 
valid  and  reliable  method  for  predicting  skill  retention  as  a  function 
of  time  and  task  characteristics.  Our  starting  point  for  analyzing  this 
work  was  the  review  by  Rose  et  al.  (1981).  However,  we  often  went  to 
the  original  sources  because  the  review  was  not  detailed  enough  for  our 
analysis,  And,  sometimes  we  disagreed  with  the  conclusions  in  Rose  et 
al.  We  did  find  their  topic  headings  useful  for  organizing  our  own  data 
base,  e.g.  Training  Methods:  Effects  of  Overtraining. 

c.  Additional  points  of  departure  were  provided  by  the  reviews  of 
Farr  (1986)  and  Schneider  (1985) .  Farr  was  a  particularly  fruitful 
source  of  information  and  conclusions,  but  here  again  as  with  Rose  et 
al.,  we  found  it  necessary  to  go  to  the  originals  for  the  details  needed 
for  our  analysis  and  applications.  And,  again,  we  borrowed  topic 
headings. 

d.  To  actually  do  the  analysis  we  reviewed  the  materials  above  and 
then  filled  out  the  data  forms  described  in  Section  2.1.b.  These 
completed  forms  constitute  our  raw  data.  For  archival  reasons,  we've 
included  them  as  Appendix  2.  From  them  we  prepared  Section  3.0,  Results. 
We  reorganized  the  materials  and  conclusions  therefrom  under  the 
categories  of  instructional  design  variables  shown  below.  To  arrive  at 
the  category  labels  ,  we  examined  those  used  by  Rose,  Farr,  and 
Schneider,  and  then  extracted  a  set  of  labels  which  seemed  to  cover 
those  reviews  in  a  frugal  way: 
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(1)  Instructional  Strategy 

(2)  Task  Characteristics 

(3)  Training  Methods 

(a)  Training  Evaluation  and  Standards 

(b)  Conditions  of  Practice 

(c)  Conditions  Which  Increase  Understanding 

2.4  Development  of  Guidelines  for  Revising  63W10  Training 

a.  Generation  of  list  of  rules.  We  examined  in  detail,  the  results 
of  this  study  including  our  description  of  the  Harless  method  (Appendix 
1)  and  the  revisions  in  the  method  suggested  to  us  by  the  scientific 
literature  (Appendix  2).  We  also  examined  Macpherson  et  al.  (1988a)  and 
Dressel  (1980).  We  literally  read  through  all  these  materials  as  a 
group  of  three,  and  extracted  rules  of  instructional  design  and 
delivery.  The  preliminary  result  was  a  list  of  60  items.  We  then 
condensed  these  into  10  major  principles  by  combining  related  or 
subordinate  items  and  eliminating  those  which  were  redundant.  The  final 
result  is  presented  in  Section  5.0. 

b.  Selection  of  test-bed  task.  We  selected  candidate  test  bed  tasks 
from  a  study  by  Macpherson,  Patterson,  and  Mirabella  (1988a).  These 
tasks  are  subject  to  very  high  rates  of  skill  decay.  They  include: 

(1)  Diagnose  stalling  engine 

(2)  Diagnose  engine  which  cranks,  but  doesn't  start 

(3)  Replace  fuel  injector 


We  will  describe  the  application  of  the  guidelines  to  these  tasks  in  a 
separate  report  (Macpherson,  Patterson,  and  Mirabella,  1988b) . 


3 . 0  RESULTS 


3.1  Maintenance  Performance  Problems  and  Current  Training  Practice 

a.  Performance  Problems.  Mirabella  (1988)  has  extensively  reviewed 
25  years  of  Army  sponsored  studies  of  maintenance  performance.  We  have 
summarized  some  of  the  more  recent  studies  below.  The  studies  have  been 
consistent  in  showing  high  rates  of  performance  failure  across  different 
testing  conditions. 

(1)  For  a  sample  of  16  classes,  average  failure  rates  in  Phase  1 
(Basic  Knowledge  and  Skills)  of  the  63W10  course  ranged  from  5  to  31%, 
across  seven  instructional  blocks  (annexes),  e.g.  for  the  annex  on 
electrical  theory,  the  average  number  of  people  failing  the  end  of  block 
test  was  31%.  The  range  of  failure  rates  across  16  classes  was  9  to  76% 
(Ramsay  et  al.,  1988a). 

(2)  For  a  sample  of  8  classes,  average  failure  rates  in  Phase  2  of 
the  63W10  course,  were  17  and  21%  for  the  hydraulics/  electricity  (E/H) 
block  and  the  end  of  course  written  exam,  respectively,  i.e.  for  the 
E/H  block  the  average  failure  rate  was  17%,  with  failure  rates  ranging 
from  7  to  30%  across  8  classes.  For  the  end  of  course  written  test  the 
average  failure  rate  was  21%,  with  failure  rates  ranging  from  7  to  50%. 
(Ramsay  et  al.,  1988a). 

(3)  In  a  TRADOC  evaluation  of  63W  and  63B  mechanics  in  FORSCOM  units, 
failure  rates  across  five  tasks  ranged  from  85  to  99%  (Ramsay  et  al. 
1988a) . 

(4)  Kern  and  Hayes  (1983)  showed  widespread  failures  among  236 
organizational  wheel  vehicle  mechanics,  on  5  different  vehicles,  e.g 
71%  of  the  mechanics  made  one  or  more  serious,  uncorrected  errors 
involving  special  tools  or  specifications,  per  maintenance  activity. 

It's  noteworthy  that  experienced  mechanics  consulted  the  TMs  for  1%  of 
the  tasks,  and  inexperienced  mechanics  used  the  manuals  for  12%  of  the 
tasks.  But  even  where  TMs  were  used,  failure  rates  were  still  very 
high.  Of  the  mechanics  who  did  use  TMs  (or  other  sources  of 
information)  ,*  66%  made  one  or  more  procedural  errors  per  task  compared 
to  79%  for  mechanics  who  did  not  consult  a  source  of  information  to 
complete  a  task.  [Incidentally,  Swezey  (1987)  verifies  that  TMs  don’t 
guarantee  successful  performance.  Novices  with  specially  prepared  job 
aids  were  able  to  outperform  experienced  electronic  mechanics  with  only 
TMs). 

(5)  The  study  by  Macpherson  et  al.  (1988a)  showed  that  skill  decay 
for  three  tasks  (listed  in  Section  2.4b)  considered  critical  by  USAOCS 
subject  matter  experts  (SMEs)  is  predictably  very  severe,  given  the 
current  training  practices. 
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b.  Current  Training  Practice.  Current  training  practice  has  been 
documented  by  Ramsay  et  al.  (1988a  and  1988b)  and  by  Macpherson  et  al. 
(1988a).  In  the  63W10  program,  basic  skills  and  knowledge  are  taught 
(BK&S)  during  Phase  1.  The  training  approach  is  to  lecture,  usually  for 
50  minutes  and  provide  group  practical  exercises  between  some  of  those 
lectures.  Groups  range  from  2  to  5  students.  An  example  of  a  PE  is  the 
assembly  and  disassembly  of  a  lawn  mower  engine  by  pairs  of  students. 
Another  example  is  an  exercise  on  how  to  set  up  and  use  the  STE/ICE  for 
electrical  measurements.  In  this  case  groups  of  five  are  used.  In  Phase 
2,  MOS-specific  tasks  are  trained,  again  with  a  mixture  of  lecture  and 
practical  exercise,  but  an  emphasis  on  the  PE. 

3.2  Research  Literature  Analysis 

The  detailed  results  of  this  analysis  are  shown  in  Appendix  2.  The 
Appendix  has  a  column  of  findings  which  are  organized  under  training 
strategy,  task,  training  methods,  measurement,  and  motivational 
variables.  These  categories  and  a  number  of  sub-categories  were  derived 
from  reviews  by  Farr  (1986),  Rose  et  al.  (1981)  and  Schneider  (1985).  A 
second  related  column  comments  on  the  relationship  between  the  findings 
and  the  validity  of  training  development  practice  and  provides  ideas  for 
improving  that  practice,  especially  as  it  is  implemented  by  the  Harless 
Workshop.  For  reasons  mentioned  in  Section  1.1,  we  are  using  the 
Harless  Workshop  as  a  representative  of  the  systems  approach  to 
training  (SAT) .  The  findings  from  this  effort  are  presented  below  with 
the  citations. 

a.  Instructional  Strategy 

(1)  Definition.  Top-level  decisions  and  philosophy  about  how  to 
organize  and  orchestrate  instruction  is  what  we  mean  by  instructional 
strategy.  The  three  most  top-level  issues  are  student-centered  vs. 
instructor-centered,  content-oriented  vs  performance-oriented,  and 
individualized  vs  cooperative  learning.  Most  if  not  all  the  research  on 
instructional  strategies  can  be  subsumed  by  these  three  issues. 

(2)  Findings. 

(a)  The  research  overwhelmingly  supports  methods  which  are 
student-centered  and  performance  oriented  (Walberg,  1986) .  But  we 
need  to  qualify  this  conclusion.  Some  recent  research  has  shown  that 
a  workshop  approach  which  uses  well-designed,  live,  group 
instruction,  followed  by  more  individualized  (i.e. student-centered) 
techniques  may  be  the  most  effective  over-all  strategy  (Woodward, 
Carnine,  &  Gersten,  1988).  Rosenshine  and  Stevens,  (1986)  suggest 
that,  in  a  non-CAI  classroom,  50%  of  the  time  (24  minutes  in  a  50 
minute  class)  should  be  used  for  demonstration  and  teacher  led 
exercises. 

(b)  Research  also  strongly  supports  the  value  of  cooperative 
learning  (Spurlin,  Dansereau,  Larson,  &  Brooks,  1984),  including  the 
cooperative  use  of  computer-based  instruction  (CBI)  (Shlechter, 1986) . 
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Cooperative  learning  is  particularly  effective  in  reducing  the  costs 
of  CBI,  because  it  can  reduce  the  number  of  workstations  and 
instructor  time  by  at  least  60%,  as  well  as  the  amount  of  time  to 
learn  by  about  30%.  But  within  these  strategies  there  are  many, 
more  specific  issues  that  need  to  be  resolved.  These  are  addressed 
in  the  remainder  of  the  results  section. 

(3)  Training  Development  Implications.  The  research  cited  above 
supports  the  Army's  System  Approach  to  Training  (SAT)  as  described  in 
TRADOC  Regulation  71-2,  TRADOC  PAM  600-11,  and  TRADOC  PAM  351-4 (T) .  It 
also  supports  the  way  SAT  has  been  implemented  by  the  TRADOC  sponsored 
Harless  Workshops  on  Front-End  Analysis,  Job  Aiding,  and  Course 
Development. 


b.  Task  Characteristics 

(1)  Definition.  To  develop  effective  training,  you  must  first 
analyze  the  characteristics  of  the  tasks  to  be  trained.  The  results  of 
this  analysis  can  then  be  used  to:  (a)  prioritize  the  tasks,  given  that 
training  resources  and  time  are  always  limited,  (b)  design  the 
courseware,  and  (c)  determine  when  refresher  training  may  be  needed.  In 
this  section  we'll  summarize  research  which  now  permits  us  to  do  (a)  and 
(c) .  In  subsequent  sections  we'll  talk  about  (b) ,  the  connections 
between  task  characteristics  and  courseware  design. 

(2)  Findings.  In  1979,  the  Army  Research  Institute  (ARI)  undertook  a 
large-scale,  long-range  program  to  study  skill  retention  and  transfer.  A 
significant  part  of  that  dealt  with  the  connection  between  task 
characteristics  and  the  soldier's  ability  to  retain  his  or  her  skill  in 
doing  the  task.  (Shields,  Goldberg,  and  Dressel,  1979a).  Shields  et  al. 
(1979a)  developed  equations  to  predict  skill  retention  over  12  months 
from  2  variables:  number  of  steps  in  the  task  and  number  of  natural  sub¬ 
groups  of  steps.  Subsequently,  this  early  work  was  extended  and  resulted 
in  a  much  more  accurate  predictive  model  based  on  10  task 
characteristics  (Rose,  Czarnolewski,  Gragg,  Austin,  Ford,  Doyle,  and 
Hagman,  1985) . 

(3)  Training  implication.  The  foregoing  research,  culminating  in  a 
validated  method  for  predicting  skill  retention,  provides  a  tool  to 
support  TRADOC  PAM  351-4 (T).  The  pamphlet  directs  you  to  prioritize 
tasks  on  the  basis  of  how  quickly  they  will  be  forgotten.  The  Skill 
Retention  Model  lets  you  solve  this  problem  in  a  quantitative  way.  This 
model  can  therefore  be  used  by  TRADOC  schools  to  help  screen  tasks  for 
inclusion  in  courses. 


c.  Training  Methods 

We've  talked  so  far  about  selecting  a  general  strategy  or  philosophy 
of  instruction  (i.e.  student  vs. instructor  centered),  and  about 
analyzing  and  selecting  tasks  for  training.  The  next  hurdle  is  to  define 
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specific  methods  of  instruction  which  will  lead,  not  only  to  effective 
learning,  but  to  retention  and  transfer  of  skills  as  well.  The  list  of 
training  method  variables  is  large,  but  we  can  organize  it  into  a  few 
broad  categories:  use  of  performance  evaluation  data  and  standards; 
conditions  of  practice  (e.g.  amount,  overlearning,  variety,  and 
spacing);  conditions  that  increase  understanding  (i.e.  "cognition");  and 
motivation  of  students. 

(1)  Training  Evaluation  and  Standards. 

(a)  Definition.  We  usually  think  of  training  evaluation  and 
standards  as  serving  the  accounting  function  of  tracking  or 
certifying  student  progress.  But  they  can  also  be  part  of  the 
instructional  methodology  per  se  and  therefore  used  to  influence 
skill  acquisition,  retention,  and  transfer. 

(b)  Findings. 

Farr  (1986,  Page  64),  for  example,  has  shown  that  you  can 
improve  retention  just  by  raising  the  criterion  for  passing 
tests,  e.g.  from  the  conventional  70%  to  80  or  85%  Based  on  his 
analysis  of  Slamecka  et  al.  (1983)  and  Loftus  (1985),  Farr  concluded 
that  the  rate  of  skill  decay  from  any  given  level  of  retention 
decreases  as  initial  skill  (i.e.  criterion)  level  increases. 

TE&S  can  also  be  used  to  support  the  use  of  feedback,  i.e. 
knowledge  of  training  results  or  reinforcement,  a  well  established 
technique  of  effective  training.  Continuous  measurement  of 
performance  can  provide  feedback  on  accuracy  of  performance.  Such 
feedback  contributes  to  both  acquisition  and  retention  of  skills 
(Schneider,  1985). 

Finally  TE&S  can  help  training  developers  refine  training 
methods.  Shields  et  al.  (1979b),  for  example,  found  that  soldiers 
tend  to  fail  the  same  steps  within  tasks.  TE&S  can  identify  such 
problem  steps  and  help  trainers  focus  resources  on  those  steps. 

On  the  other  hand  research  by  Schneider  (1985)  shows  that 
TE&S  can  be  misapplied.  Schneider  concluded  from  his  research  on 
skill  development  that  students  should  not  be  screened  out  because 
they  perform  poorly  at  the  start  of  a  long  training  program  since 
early  test  results  do  not  predict  final  success.  TE&S  can  be 
misapplied  in  another  way. 

Go/no  go  measures  on  an  entire  task  with  many  steps  can 
give  misleading  information  about  skill  decay,  if  only  one  or  two 
steps  account  for  the  failures.  This  is  particularly  true  for  the 
situations  described  by  Shields  et  al.  (1979b)  where  the  same  few 
steps  are  consistently  failed  by  many  trainees.  For  example,  in  a 
study  done  by  TRADOC'S  Training  Analysis  Command  (TRAC),  91  out  of  98 
Army  mechanics  failed  a  test  in  brake  overhaul  (Noel,  1987).  But  TRAC 
later  showed  that  this  high  failure  was  caused  by  one  misstep, 
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failure  to  torque  the  bolts  in  replacing  vehicle  wheels.  And  even 
this  misstep  was  an  artifact  of  the  test  procedures. 

(c)  Training  Implications. 

Training  Standards.  The  Army  can  improve  skill  retention  of 
its  school  graduates  by  raising  test  standards  from  the  traditional 
(in  some  schools)  70%  to  a  higher  level  -  perhaps  85  or  90  %.  This 
change  would  increase  demands  on  remedial  training  and  recycling,  but 
EIDs  or  other  high  technology  training  media  can  provide  an  efficient 
way 'to  cope  with  the  added  demands. 

The  Army  can  further  increase  the  efficiency  of  its 
training  by  using  evaluation  data  to  identify  the  most  difficult 
parts  of  tasks,  and  focusing  extra  resources  on  those  parts,  fewer 
resources  on  less  difficult  parts.  At  the  same  time  it  needs  to  avoid 
misusing  evaluation  data.  Go/no  go  measures  of  over-all  task  success 
need  to  be  used  cautiously.  More  refined  measures  should  be 
considered  especially  to  provide  knowledge  of  results  to  trainees  and 
to  help  training  developers  improve  instruction.  A  good  example  of 
high  quality,  fine  grained  Army-relevant  measurement  is  provided  by 
the  work  of  Applied  Sciences  Associates  (1988)  for  the  TTFA-Aberdeen . 
We'll  develop  some  of  these  ideas  in  the  next  section  on  conditions 
of  practice. 

The  use  of  evaluation  data,  described  in  the  preceding 
paragraph  fits  well  into  the  Harless  Method,  since  it  tells  you  to 
identify  problem  tasks  and  steps  within  tasks  before  you  develop 
courseware . 

(2)  Conditions  of  Practice. 

(a)  Amount  of  Practice  Leading  to  Initial  Proficiency. 

Findings.  It's  well  established  that  learning  increases 
with  practice  until  performance  reaches  a  ceiling.  But  the  old  adage 
"practice  makes  perfect"  can  actually  lead  to  inefficient  training 
(Shields,  Goldberg,  and  Dressel,  1979a;  Schneider,  1985).  Shields  et 
al.  (1979a)  found  that,  for  many  tasks,  soldiers  consistently  failed 
only  a  few  of  the  same  steps.  Schneider  corroborated  this  finding 
for  air  traffic  control.  And  he  noted  that  when  students  repeatedly 
practiced  whole  tasks,  they  overtrained  on  frequently  occurring 
components  of  the  tasks,  without  learning  less  frequently  or 
especially  critical  components. 

Training  Implication:  Use  part  task  training  methods  to 
provide  extensive  practice  on  hard  or  especially  critical  task  steps. 
Programmed  (including  computer-aided)  instruction  can  be  very 
efficient  for  this  purpose.  The  Harless  method  is  especially  well 
designed  to  provide  the  proper  amounts  of  part  and  whole  task 
training  since  it  breaks  tasks  into  small,  meaningful  parts,  and  then 
rolls  them  up  into  integrated  exercises,  finally  ending  in  a 
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simulation  of  the  whole  task. 

(b)  Variety  of  Training  Materials. 

You  can  increase  training  effectiveness  by  using  a  variety 
of  materials  or  by  varying  the  conditions  of  training.  Doing  so  will 
improve  both  skill  retention  and  transfer  to  new  situations,  but  this 
method  will  especially  improve  transfer.  It  works  both  for  initial 
skill  acquisition,  which  we  are  talking  about  here,  and  for 
overtraining  which  we  will  talk  about  in  the  next  section  (Battig, 
1979;  Shea  and  Morgan,  1979;  Farr,  1987).  In  particular,  Schneider 
(1985)  recommends  that  you  should  vary  those  aspects  of  tasks  and 
conditions  which  may  vary  on  the  job.  Keller  (1983)  recommends  the 
use  of  variety  to  maintain  motivation. 

But  too  much  variety  introduced  too  quickly  can  have  a 
negative  side-effect.  It  can  confuse  students  and  increase  the 
difficulty  of  initial  learning.  Bransford  et  al.  (1979)  recommend 
that  you  use  the  same  or  similar  materials  for  a  block  of  practice 
sessions  before  varying  task  characteristics  or  conditions  in 
subsequent  blocks.  In  addition  you  may  need  to  emphasize  similarities 
or  differences  in  the  varied  materials  (Engelmann  and  Carnine,  1982 
Brooks,  1988;  Perkins  and  Simmons,  1988). 

(c)  overtraining:  Practice  Beyond  Initial  Proficiency. 

Practice  beyond  proficiency  can  significantly  improve  skill 
retention  (Rose  et  al,  1981).  In  fact  overtraining  at  the  school 
house  can  reduce  the  need  for  refresher  training  in  the  unit  and 
thereby  reduce  the  cost  of  such  training  (Schendel  and  Hagman,  July, 
1980;  Rose,  McLaughlin,  and  Felker,  1981).  Moreover,  the  principle 
works  for  a  wide  variety  of  tasks:  M60  assembly/disassembly 
(Schendel  and  Hagman,  July,  1980),  boresighting  (Goldberg  et  al., 
1981),  and  alternator/generator  testing  (Hagman,  May  1980a). 

Three  repetitions  of  a  task  beyond  the  priming  stage  of 
training  (i.e.  familiarization,  explanation,  illustration)  are 
optimal  for  maintaining  skills  (Hagman,  May,  1980a) .  But  to  maximize 
transfer  to  new  situations  as  well  as  skill  retention,  you  need  to 
vary  the  practice  sessions  (November,  1980b) .  For  example,  if  you 
were  teaching  soldiers  to  test  alternators,  you  would  have  the 
soldiers  practice  on  three  different  types,  rather  than  repeat 
practice  on  one  type.  In  fact  overtraining  on  the  same  equipment  can 
actually  interfere  with  ability  to  handle  new  situations.  One 
further  refinement  on  the  principle  of  overtraining  is  necessary  to 
maximize  the  effectiveness  of  training:  Practice  sessions  should  be 
spaced  (i.e.  separated  by  "rest"  periods). 

(d)  Spacing  of  Practice  Sessions.  Long-standing  basic  research 
has  shown  that  if  students  "rest"  between  practice  sessions,  they 
will  retain  what  they've  learned  much  better.  Hagman  (1980b)  verified 
this  principle  for  military  training.  He  found  that  if  he  separated 
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practice  sessions  (during  overtraining)  by  one  day  and  used 
different  versions  of  the  sane  equipnent  (i.e.  varied  the  training 
naterials) ,  he  could  dramatically  improve  both  skill  retention  and 
transfer  to  new  equipment. 

(e)  Presentation/Explanation  vs.  Practice.  The  50-minute  stand- 
up  lecture  continues  to  be  the  dominant  instructional  method  both  in 
and  out  of  the  military.  But  the  bulk  of  evidence  indicates  that 
students  who  practice  individually  at  least  part  time  learn  and 
retain  skills  more  effectively  than  those  subjected  full-time  to 
lectures.  (Rosenshine  and  Stevens,  1986;  Swezey,  Perez,  and  Allen, 
1988;  Woodward,  Caraine,  and  Gersten,  1988).  And  for  individual 
practice,  a  mixture  of  prompted  and  unguided  practice  is  more 
effective  than  either  alone  (Hagman,  January,  1980) ,  especially  if 
unguided  practice  includes  some  discovery  learning  (Wellman  and 
Hagman,  1988) .  Discovery  learning  means  that  instead  of  "spoon¬ 
feeding"  correct  answers,  you  ask  leading  questions  which  help  the 
student  discover  the  right  answer. 

(3)  Conditions  Which  Increase  Understanding.  The  effects  of  well 
designed  practice  sessions  can  be  much  enhanced  by  using  techniques 
which  foster  the  students  "understanding"  of  the  materials  to  be  learned 
and  of  the  responses  which  must  be  made  to  those  materials.  These 
techniques  or  more  properly  categories  of  techniques  include  elaboration 
methods  and  the  proper  use  of  "theory". 

(a)  Elaboration  Methods.  Students  recall  meaningful 
materials  more  easily  than  those  which  are  less  meaningful.  And  they 
can  use  such  recalled  knowledge  more  effectively  in  new  situations. 
Elaboration  methods  are  ways  that  students  can  increase 
meaningfulness  by  associating  materials  and  behaviors  to  be  learned 
with  materials  and  behaviors  already  familiar  to  the  student.  Farr 
(1986)  mentions  the  following  methods  and  supporting  research: 

Mnemonics.  Mnemonics  are  characters,  words,  sentences,  or 
pictures  which  help  students  remember  facts,  rules,  or 
procedures.  They  take  the  form  of  similes,  metaphors  (verbal  or 
graphic) ,  acronyms,  acrostics,  rhymes  or  combinations  of  the 
foregoing.  The  Japanese  have  successfully  used  mnemonics, 
especially  metaphors,  to  teach  math,  chemistry,  physics, 
biology,  spelling,  and  English  (Higbee  and  Kunihira,  1985). 

But  a  study  on  acrostics  (to  help  soldiers  recall 
procedures  for  operating  equipment)  failed  to  support  the  value 
of  mnemonics  (Dressel,  1980)  for  training  in  the  US  Army.  (An 
acrostic  is  a  sentence  in  which  the  first  letter  of  each  word 
denotes  some  other,  more  meaningful  word) .  The  differences  here 
are  instructive.  The  Japanese  build  their  mnemonics  from  simple 
language  or  pictures,  which  are  very  familiar  to  the  students. 
Dressel* s  acrostics  were  not  simple,  e.g.  "Explosions  Usually 
Terrify  Individual  Defenders,  Although  Platoons  Rarely  Scare." 
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The  implication  for  training  is  that  you  should  consider 
using  mnemonics,  but  make  sure  they  are  not  as  difficult  as  the 
material  to  be  learned.  The  contrasting  Japanese  and  ARI 
experiences  show  that  you  need  to  make  the  mnemonic  easy  and 
tie  it  to  the  student's  existing  knowledge.  This  connection  is 
the  key  ingredient  in  any  elaboration  technique. 

Semantic  Coding.  The  research  on  semantic  coding,  cited  by  Farr 
(1986;  Page  55),  reaffirms  again  the  old  wisdom  that  meaningful 
materials  are  recalled  and  recognized  more  easily  than  less 
meaningful  materials.  In  the  semantic  coding  studies  (Mistier- 
Lachman,  1974?  Craik  and  Tulving,  1975?  Bower  and  Karlin,  1974) 
the  experimenters  presented  words  to  subjects  and  asked  them 
increasingly  "meaningful''  questions  about  the  word,  e.g.  lower 

case  vs.  upper  case  letters,  word  rhymes  with  _  ,  wird  does 

or  does  not  fit  a  particular  sentence.  The  more  "meaningful" 
the  question,  the  higher  the  recall  of  that  word  in  a  later 
memory  test. 

The  implication  for  training:  Identify  terms  that  may  be 
unfamiliar  or  confusing  to  students  (e.g.  volt  and  ampere)  and 
spend  extra  time  defining  them.  Make  up  some  sentences  using  the 
terms,  and  challenge  students  to  figure  out  whether  the 
sentences  make  sense.  Have  the  students  correct  sentences  which 
don't.  Use  each  term  once  in  a  correct  sentence  and  an  incorrect 
sentence,  then  repeat  the  exercise  with  different  sentences. 

Analogies.  The  use  of  analogies  is  perhaps  the  most  direct  and 
deliberate  way  to  connect  new  learning  to  prior  knowledge.  For 
example,  electrical  voltage  can  be  "explained"  by  comparing  it 
to  pressure  at  the  bottom  of  a  waterfall.  Analogies  can  be  very 
effective,  especially  for  complicated  subjects  like  electricity 
(Gentner  and  Gentner,  1983) .  But  students  must  first  understand 
the  analogy  itself  and  then  its  precise  connection  to  the 
material  to  be  learned.  If  they  don't,  the  analogy  will 
interfere  with  learning.  If,  for  example,  you  want  to  use 
hydraulics  to  "explain"  electricity,  make  sure  your  students 
first  understand  the  relevant  ideas  from  hydraulics.  Then 
explain  how  each  part  of  the  analogy  is  connected  to  each  part 
of  the  new  materials. 

Advanced  Organizers  fAOs) .  These  are  previews  of  more  detailed 
instruction  to  follow.  They  include  introductory  outlines, 
learning  objectives,  hints  on  what  to  look  for,  and  preliminary 
questions.  See  Gropper  (1987)  for  good  examples.  AOs  can  provide 
a  "big  picture",  a  "view  of  the  forest  before  the  trees",  a 
"skeleton"  on  which  to  hang  the  "meat"  of  instruction.  They  can 
improve  training  effectiveness  if  they  are  relevant  and  closely 
tied  to  the  main  instruction,  otherwise  they  simply  waste  time 
and  resources  (Ausabel,  1961). 
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(b)  Explanation  and  Theory. 

Definition.  We've  talked  so  far  about  the  importance  of 
designing  the  conditions  of  practice  and  about  elaborating  the 
materials  to  be  learned  to  make  their  meaning  clear.  But,  is  it 
important  to  go  further  and  get  "beneath  the  surface."  Should 
we  teach  what  has  been  loosely  referred  to  as  theory,  and  if  so 
what  should  this  instruction  look  like,  how  much  should  there 
be,  and  when  should  we  provide  it?  The  value  of  "theory"  has 
been  much  debated  and  that  debate  has  occasionally  been 
.translated  into  extreme  positions,  e.g.,  new  math  with  its 
emphasis  on  theory  at  the  expense  of  practice,  followed  by  a 
counter-reaction  against  teaching  mathematical  theory  (Howson  et 
al,  1981;  Skinner,  1989).  The  research  cited  by  Farr  (1986) 
provides  interesting,  compelling,  and  plausible  answers  to  the 
questions  about  the  role  of  theory. 

Findings.  Students  recall  and  generalize  procedures  better  if 
they  are  taught  "theory,"  i.e.,  taught  the  reasons  for  the 
procedures  and/or  how  equipment  is  built  and  operates.  If 
students  are  taught  to  understand  how  equipment  works,  they 
will: 


•  learn  operating  procedures  faster 

•  remember  them  longer, 

•  work  more  efficiently, 

•  infer  missing  information  much  faster,  and, 

•  recover  from  problems  more  easily. 

However,  the  trainer  will  get  the  above  effects  only  if  the 
theory  and  understanding  directly  support  the  specific  skills 
which  students  must  learn  in  order  to  do  a  job.  "How-it-works" 
knowledge  is  useless  and  inefficient  otherwise.  In  particular, 
the  literature  on  functional  context  training  says  that  theory 
must  support  and  be  integrated  with  practice  if  it  is  to  be 
useful  (Sticha,  1987;  Montague,  1988).  Other  literature, 
including  a  study  by  Kieras  and  Bovair  (1984) ,  a  study  by 
Alexander  et  al.  (1987)  on  general  training  in  analogous 
reasoning,  and  an  assessment  of  "new  math"  by  Howson  et  al., 
(1981)  support  this  conclusion. 

What  kind  of  theory  and  how  much  is  useful?  Schneider 
(1985)  found  that  for  air  traffic  control,  training  a  conceptual 
understanding  of  the  control  system  did  not  ensure  proficiency 
on  the  job.  Kieras  and  Bovair  (1984)  similarly  found  that 
information  about  "theory"  of  equipment  operation  was  helpful 
in  teaching  equipment  operation  procedures  only  where  the 
"theory"  was  clearly  linked  to  specific  operations.  In  fact 
full,  detailed,  and  complete  explanations  about  how  the 
equipment  worked  interfered  with  learning  the  required  task. 
Kieras  and  Bovair  found  that  "how-it-works"  knowledge  can  be 
superficial,  and  incomplete,  and  still  be  effective  in  helping 
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students  learn,  remember,  and  transfer  procedural  skills. 

Implications  for  Training.  Clearly,  rote  learning  "by  the 
numbers11  is  not  very  efficient  for  retention  and  transfer,  even 
with  elaborations  such  as  mnemonics  and  analogies. 
Instructors/training  designers  need  to  give  explanations  of 
••why"  as  well  as  "how",  i.e.  the  theory  behind  the  practice. 

But  the  theory  has  to  be  relevant  to  and  carefully  integrated 
with  task  training.  Instructors  can't  teach  "shot-gun  theory", 
hoping  that  students  will  retain  and  transfer  task  skills 
better.  They  need  to  first  define  the  skills  they  want  to  build 
in  students  and  then  decide  what  background  or  deeper 
understanding  would  support  learning  the  task  skills. 
Furthermore,  instructors  should  start  their  teaching  with  "how" 
explanations  and  weave  in  or  follow-up  with  "why"  explanations. 

At  the  Ordnance  School,  the  practice  of  teaching  basic 
knowledge  and  skills  (BKS)  first,  then  MOS  skills  illustrates 
the  wrong  way  to  teach  theory  and  is  both  inefficient  and 
wasteful.  The  practice  is  inefficient  not  only  because  BKS  and 
MOS  training  are  separated,  but  also  because  much  of  the  BKS  is 
irrelevant.  The  course  textbook,  TM  9-8000,  for  example  is  a 
text  on  automotive  engineering  that  has  little  to  do  with 
maintenance. 

The  Harless  method  minimizes  the  problem  of  misusing 
theory,  because  it  focuses  on  required  job  performance. 

However,  it  goes  to  an  extreme  here,  and  would  be  more  effective 
if  it  incorporated  some  theory  training  as  well  as  more 
elaborative  techniques.  Gropper  (1987)  illustrates  how  the 
performance  oriented  approach  can  incorporate  theory. 


d.  Motivating  the  Student.  So  far  we've  been  focusing  on  how  to 
design  instruction  which  effectively  develops  skills.  But  instruction 
cannot  be  effective,  or  at  least  efficient,  if  student  motivation  is 
not  maintained.  Can  we  design  instruction  to  maximize  its  interest  to 
the  student,  as  well  as  to  maximize  its  effectiveness  in  developing 
skills?  Yes,  .at  least  partly  because  what  makes  instruction  effective 
also  makes  it  interesting.  For  example: 

(1)  Reinforcement  as  a  Motivator.  Schneidner,  (1985)  found  that  by 
playing  an  "interesting"  sound  when  students  performed  correctly,  he 
reduced  failure  rates  in  an  otherwise  boring,  day  long,  air  traffic 
control  training  sessions  from  30  to  5%.  He  also  found  he  could  maintain 
both  interest  and  high  performance  by  insuring  that  students 
participate  actively  in  the  instruction,  through  frequent  responding,  - 
e.g.  every  few  seconds.  But,  a  more  systematic  and  comprehensive  attack 
on  the  problem  of  motivation  itself  has  been  made  by  Keller  through  the 
Attention,  Relevance,  Confidence,  and  Satisfactions  Model  (ARCS) . 
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(2)  Keller's  ARCS  Model.  From  his  analysis  of  the  research  on 
motivation,  especially  research  on  gaming  techniques,  Keller  (1983) 
identified  four  dimensions  of  motivation  that  could  be  applied  to 
instruction:  Attention,  Relevance,  Confidence,  and  Satisfaction.  He 
then  developed  instructional  techniques  for  creating  high  values  of 
these  dimensions  in  instructional  material  (particularly  computer- 
based)  . 

Keller  and  Kopp  (1987)  summarized  the  model  in  a  set  of  training 
design  prescriptions  and  applied  it  to  an  introductory  lesson  on  how  to 
use  a  microscope.  Keller  and  Suzuki  (1988)  did  the  same  thing  except 
they  detailed  and  illustrated  the  prescriptions.  They  then  applied  the 
model  to  a  computer-based  instructional  lesson  on  leadership. 

(3)  Training  Implications.  Student  motivation  is  not  often 
considered  in  training  development.  But  Keller's  prescriptions  show 
some  specific  ways  that  this  can  be  done.  He  has  shown,  in  the 
references  cited  above,  that  those  prescriptions  can  be  applied  to  CBT 
or  other  delivery  approaches,  even  lecture'.  His  ideas,  therefore,  can  be 
applied  to  a  range  of  settings  in  TRADOC  schools. 

Three  of  Keller's  dimensions  are  built  into  the  Harless  method: 
Relevance,  Confidence,  and  Satisfaction.  You  can  easily  incorporate  the 
fourth.  Attention.  Harless  provides  relevance  by  focusing  on  task 
performance  that  is  deficient  but  critical  to  a  specific  job.  He  builds 
in  confidence  by  breaking  tasks  into  small,  meaningful  units  and  then 
rolling  up  the  units  into  integrated  exercises.  He  fosters  satisfaction 
by  gradually  withdrawing  prompts  but  always  providing  feedback  to  the 
student.  He  also  fosters  satisfaction  by  having  the  student  apply  newly 
learned  skills  in  job  simulation  exercises. 


4 . 0  DISCUSSION 
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4.1  Overview  of  Findings 

a.  The  findings  show  that  many  valid  training  techniques  and 
strategies  are  available  to  help  soldiers  (1)  remember  skills  taught  at 
TRADOC  schools  and  (2)  transfer  the  skills  to  new  situations.  In  the 
absence  of  these  methods,  serious  performance  deficiencies  can  be 
expected,  and  in  fact  have  been  documented  both  in  the  school  house  and 
in  the  unit  (Kern  &  Hayes,  1983;  Macpherson  et  al.,  1988a,  1988b;  Ramsay 
et  al.,  1988a).  We've  condensed  these  techniques  and  strategies  into  10 
principles  which  we  will  explain  at  length,  along  with  their  rationales. 


b.  The  findings  also  validate  TRADOC* s  SAT  training  philosophy  and 
its  implementation  through  the  Harless  Associates  Workshop.  The  fly  in 
the  ointment  is  that  neither  the  philosophy  nor  its  specific  methods  are 
much  in  evidence  in  the  63W10  program  (Ramsay  et  al.,  1988a,  1988b). 

c.  For  example,  SAT  and  the  research  we  have  examined  clearly 
indicate  that  basic  skills  and  knowledge  (BK&S)  should  be  derived  from 
and  very  closely  linked  to  related  tasks  during  training.  The  common 
approach  to  teaching  maintenance,  and  the  one  used  at  APG,  is  to  teach 
BK&S  first,  as  a  course  in  engineering,  rather  than  maintenance  and  then 
separately,  at  a  substantially  later  time,  to  provide  shop  practice  in 
the  MOS  tasks. 

d.  This  ineffective  and  wasteful  approach  is  aggravated  by  the 
dominant  use  of  stand-up  lectures,  in  place  of  a  workshop  routine  which 
could  capitalize  on  the  widely  accepted  (among  professional  trainers  and 
instructional  psychologists)  5-P  principle:  prime,  prompt,  perform, 
isolated  practice,  integrated  practice.  We  will  explain  this  in  the 
next  section. 

e.  We  were  glad  to  find  that  much  of  the  recent  applied  research  on 
skill  retention  and  transfer  supports  principles  that  have  been  well  and 
long  established  by  basic  research.  In  particular  the  value  of 
overtraining,  especially  if  it  is  distributed  (spread  out)  over  several 
days  and  on  different  pieces  of  equipment  has  been  reliably  established. 
This  is  not  common  practice  at  APG,  possibly  because  it's  time 
consuming.  But  it  could  be  applied  to  a  few  tasks  that  are  particularly 
critical  and  subject  to  severe  skill  decay.  And,  this  practice  is  one 
answer  to  the  often  heard  complaint  that  mechanics  who  learn  on  one 
piece  of  equipment,  may  fail  the  same  task  on  another  piece  of 
equipment. 
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4.2  Training  Issues  Which  Are  Not  Well  Resolved 

a.  The  research  as  well  as  Harless  suggest  that  long  tasks  are 
easily  forgotten,  but  that  the  skill  decay  can  be  substantially  reduced 
if  they  are  divided  into  meaningful  sub-tasks  which  are  labeled  and  then 
taught  as  separate  exercises.  Conclusive  evidence  to  support  such  a 
practice  would  be  very  useful. 

b.  The  value  of  mnemonics  and  analogies  are  well  established.  But 
how  do  you  design  them  for  particular  audiences  and  topics?  We  need 
research  to  help  us  write  specifications  for  producing  and  using  these 
techniques . 

c.  Our  review  has  cited  some  evidence  that  you  can  maximize 
instructional  effectiveness  by  combining  live  group  instruction  with 
individualized,  particularly  computer  aided,  training.  But  the  number 
of  studies  on  when  and  how  to  combine  these  two  instructional  delivery 
approaches  is  small.  We  need  to  more  extensive  evidence  fron  applied 
research  on  how  best  to  integrate  theory  and  practice.  Much  of  what  we 
know  about  the  issue  of  integration  comes  from  small  basic  studies. 
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5.0  PRINCIPLES  FOR  IMPROVING  SKILL  ACQUISITION 
RETENTION  AND  TRANSFER 


Prologue:  If  you  use  these  principles  you  will  reduce  net  training  costs 
and  increase  unit  readiness  by  increasing  skill  retention  and  transfer 
to  new  equipment.  Even  if  you  use  a  few  of  them  or  apply  them  less 
strictly,  you’ll  see  major  improvements  in  learning  and  performance. 
Since  many  of  the  principles  may  demand  extra  resources,  apply  them  to  a 
prioritized  list  of  tasks.  For  high  priority  tasks,  apply  all  the 
principles  "full  strength.”  For  lower  priority  tasks,  use  some  of  them 
or  scale  them  back. 


5.1  Identify  and  prioritize  problem  tasks. 

a.  Rationale:  You  can  train  more  efficiently,  if  you  apply  our 
guidelines  to  the  most  critical  tasks  which  you  know  are  hard  to  learn, 
easily  forgotten,  or  are  not  transferred  well  to  new  situations. 

b.  How-to:  Use  test  and  field  evaluation  data,  instructor 
experience  with  the  students,  and  the  ARI  Skill  Retention  Model  (Ramsay 
et  al.,  1988a;  Macpherson  et  al.,  1988a). 


5.2  Identify  steps  within  tasks  that  are  difficult  or  are  performed 
incorrectly  by  soldiers  and/or  trainees. 

a.  Rationale:  Both  research  and  evaluation  data  tell  us  that 
soldiers  often  fail  to  perform  tasks  because  they  have  trouble  with 
some  steps,  not  with  the  entire  task.  Below  we've  identified  some 
special  problems  and  what  to  do  about  them. 

(1)  "Red  flag"  steps  where  soldiers  have  to  choose  among 
alternative  courses  of  action  (i.e.  discrimination)  or  where  the  same 
or  similar  actions  will  fit  different  conditions  (i.e.  generalization). 
These  parts  of  tasks  need  special  attention  and  training  strategies. 

For  example  you  may  have  to  break  them  into  smaller  steps.  For 
difficult  discriminations,  start  with  easy  examples  and  then  gradually 
use  more  difficult  ones.  For  generalizations  provide  practice  in  a 
variety  of  situations,  e.g.  practice  troubleshooting  an  alternator  on 
three  different  vehicles. 

(2)  Safety  procedures  are  frequently  violated  (Shields  et  al., 

1979a;  Kern  &  Hayes,  1983).  One  reason  is  that  these  involve 
discriminations  about  how  "safe  is  safe.”  But  they  are  listed  as  simple 
procedures,  not  as  discriminations  in  the  usual  task  analysis. 
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(3)  "Red  flag"  steps  where  it's  hard  to  tell  whether  the  step  was 
performed  successfully.  You  need  to  teach  what  might  go  wrong  and  the 
signs  of  successful  vs.  unsuccessful  performance.  The  Kern  and  Hayes 
(1983)  study  showed  that  this  can  be  a  big  problem. 

(4)  Identify  tasks  that  require  automated  (i.e.  instinctive) 
actions,  e.g.  tasks  performed  under  very  severe  time  pressure  and  where 
the  soldier  has  to  track  and  recall  5  or  more  items  of  information. 


5.3  Integrate  basic  skills  and  knowledge  (6K&S)  instruction  with  MOS 
task  training. 

a.  Rationale:  BK&S  training  which  teaches  "shotgun"  theory  about  the 
construction  and  operation  of  automotive  power  plants  is  very 
inefficient,  very  wasteful,  and  could  even  interfere  with  transfer  of 
training  rather  than  improve  it.  Research  has  clearly  shown  that 
background  material,  i.e.  explanations  about  how  equipment  works,  is 
useful  only  if  it's  closely  tied  to  the  actual  tasks  which  people  have 
to  perform. 

b.  How-to: 

(1)  Combine  related  annexes  from  Phases  1  (BK&S)  and  2  (MOS  Task 
Training) . 

(2)  Derive  BK&S  from  the  task  analyses.  Ask  yourself  how  much  and 
what  kind  of  BK&S  is  relevant  to  the  task.  Teach  only  that.  Caution: 
This  is  not  a  prescription  for  rote  training.  Mechanics  have  to 
understand  how  equipment  works  and  why  tasks  and  task  steps  are 
performed.  But  you  need  to  tie  explanations  and  task  procedures  closely 
together. 


5.4  Replace  stand-up  lecture  with  workshop  format. 

a.  Rationale:  The  stand-up  lecture  violates  a  basic  rule  of  good 
instruction:  "Learn  by  doing".  The  traditional  lecture  method  is  at 
least  50  years  behind  the  times. 

b.  How-to:  Mix  lecture  with  group  and  individual  exercises.  Limit 
lecture  to  20  minutes,  other  activities  to  30  minutes. 


5.5  Use  the  fundamental  principle  of  good  instruction,  the  5-Ps. 

a.  Prime:  Explain  and/or  demonstrate. 

b.  Prompt:  Students  practice  part  tasks  with  help. 

c.  Perform:  Students  practice  part  tasks  with  reduced  help. 
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d.  Isolated  Practice:  Students  practice  the  whole  task  without  help. 

e.  Integrated  Practice:  Students  practice  current  task  with 
previously  learned  tasks. 


5.6  Raise  test  standards  and  improve  hands-on  testing. 

a.  Rationale:  Testing  is  usually  thought  of  as  a  way  to  qualify 
people.  But,  good  testing  is  also  an  effective  training  tool.  For 
example,  you  can  reduce  skill  decay  by  training  soldiers  to  a  higher 
level  of  achievement.  Good  testing  can  also  provide  a  " thermometer"  to 
judge  the  "health”  of  a  training  program.  It  can  give  you  clues  to 
where  and  how  to  improve  the  POI. 

b.  How  to: 

1)  Raise  the  current  testing  standard  from  70%  to  85%.  Make  liberal 
use  of  remedial  training  to  achieve  this  standard.  Incidentally,  here 
is  a  good  place  to  use  high  tech  methods  such  as  the  Computerized 
Handheld  Instructional  Prototype  (CHIP) ,  Electronic  Instructional 
Delivery  System  (EIDS) ,  or  other  forms  of  computer  assisted  instruction. 

2)  Improve  hands-on  testing  by  having  students  go  through  all  test 
stations  at  the  end  of  Phase  2  of  the  63W10  POI.  The  current  tests  are 
not  valid  because  they  limit  students  to  one  station  out  of  several. 
Students  are  so  limited  because  of  the  time  required  to  test.  But  a 
compromise  technique  is  to  test  on  parts  of  tasks. 


5.7  Sequence  training  from  overviews  to  specifics,  from  concrete  to 
abstract,  and  from  familiar  to  unfamiliar  materials. 

a.  Rationale:  An  overwhelming  amount  of  both  basic  and  applied 

research  tells  us  that  people  learn  most  effectively  and  can  retain  and 

transfer  skills  best  if  they  can  tie  new  learning  to  what  they  already 

know.  Each  kind  of  sequence  mentioned  above  is  designed  to  put  this 

research  finding  to  use. 

# 

b.  How-to: 

(1)  Use  advanced  organizers  to  help  students  separate  the  "forest 
from  the  trees",  e.g.  outlines,  flow  charts,  overviews  of  equipment  or 
systems.  Advanced  organizers  can  help  students  understand  how  and  why 
different  parts  of  a  task  fit  together  or  how  different  tasks  fit 
together  to  accomplish  some  larger  purpose.  They  can  also  show  how 
equipment  construction  and  operation,  and  maintenance  practices  mesh 
with  one  another. 

(2)  Then  train  concrete  tasks  on  simplified  everyday  equipment  from 
the  student's  experience  (e.  g.  use  flashlight  batteries,  hook-up  wire. 
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a  simple  on-off  switch,  and  miniature  light  bulbs  to  introduce 
electrical  troubleshooting) . 

(3)  Train  on  actual  equipment. 

(4)  Train  on  theory  required  to  perform  the  task. 

(5)  Train  on  different  type  of  actual  equipment.  Have  students 
apply  theory. 

(6)  When  training  students  to  perform  a  task  on  several  types  of 
equipment  train  on  the  additional  equipment  that  is  most  like  the  first, 
then  train  on  dissimilar  equipment. 

(7)  Train  from  students'  experiences,  what  they  know,  to  what  they 
don't  know.  Use  analogies  and  examples  that  the  students  already 
understand. 


5.8  Use  specialized  part-task  training  techniques. 

a.  Rationale:  Breaking  a  task  into  smaller  pieces  and  teaching  the 
pieces  can  be  a  very  effective  way  reduce  skill  decay.  Fortunately,  the 
research  on  skill  retention  has  provided  some  practical  rules  (and 
reasons)  for  doing  this. 

b.  How-to: 

(1)  Break  long  tasks  (more  than  10  steps)  into  smaller  groups  of 
steps  and  teach  each  group  as  a  separate  training  exercises,  e.g.  for  a 
12-step  task,  separate  it  into  two  groups  with  6  steps  each.  Use 
natural  groupings  and  give  them  meaningful  labels. 

(2)  Train  students  to  recognize  successful  completion  of  each  step, 
especially  where  tasks  don't  provide  any  natural  feedback  on  success  or 
failure. 

(3)  Provide  extra  practice  for  difficult  steps,  long  sub-tasks  that 

could  not  be  divided  further,  discriminations,  and  generalizations. 

# 

(4)  Use  highlighting  techniques  to  help  students  focus  on  and 
understand  training  materials,  especially  for  abstract  materials  and 
tasks  without  built-in  feedback,  e.g  special  emphasis,  mnemonics, 
analogies,  probe  questions.  Use  the  techniques  correctly  or  don't  use 
them  at  all. 

(5)  Special  Rules  for  mnemonics: 

(a)  Use  familiar  words  and  phrases  which  the  student  can 

understand  and  recall  easily  with  no  additional  training. 

(b)  Use  short  words,  one  or  two  syllables. 
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(c)  Use  mnemonics  only  for  recalling  words  or  short  phrases. 
Mnemonics  don't  work  well  for  recalling  lists  of  complex  procedures. 


(6)  Special  rules  for  analogies 

(a)  Make  sure  the  students  understand  the  analogy,  e.g.,  if  you 
want  to  use  hydraulics  as  an  analogy  for  electricity,  teach 
hydraulics  first,  and  then  check  the  students'  understanding. 

(b)  Ensure  that  both  the  elements  of  the  analog,  e.g., 
hydraulics,  and  the  relations  among  the  elements  are  similar  to  those 
of  the  subject  matter  being  taught.  Student  must  clearly  see  or  be 
shown  the  similarity. 

(c)  For  your  benefit  as  well  as  the  students',  prepare  a  table 
showing  the  comparable  elements  and  relationships  of  the  analog  and 
the  material  to  be  learned.  See  the  diagram  of  electrical  analogies 
on  page  50. 

(d)  Different  analogies  may  be  needed  for  different  parts  of  a 
topic,  e.g.,  a  pump  pushing  water  through  a  narrowed  pipe  is  a  good 
analogy  for  explaining  the  effect  of  series  resistance  on  voltage  in 
an  electrical  circuit,  but  is  not  effective  in  explaining  the  effects 
of  parallel  resistors.  A  "moving  crowd"  analogy  works  better  in  the 
latter  case  (horse  running  around  a  track  and  suddenly  coming  to  a 
pair  of  parallel  gates) . 

(e)  Bottom  line:  An  analogy  can  be  very  helpful,  but  be  sure 
the  connection  between  it  and  the  material  to  be  learned  is  clear  to 
your  students.  If  in  doubt,  try  out  2  or  3  different  analogies  and 
check  your  students'  understanding  of  the  analogy. 

(7)  Decrease  help  as  students  gain  skill  on  a  task. 


5.9  Train  for  both  retention  and  ability  to  work  on  new  equipment 

a.  Rationale:  It's  important  for  students  to  remember  the  specific 
task  training  they  received,  but  they  also  have  to  be  prepared  to  meet 
new  situations,  e.g.,  new  equipment.  Basic  and  applied  research  provide 
some  ideas  that  seem  to  work  dependably. 

b.  How-to: 

(1)  Train  each  task  on  a  specific  equipment  to  the  point  that  the 
student  can  perform  it  once  unaided. 

(2)  Have  the  student  perform  the  same  task  three  more  times  on  2  or 
3  different  pieces  of  equipment.  Spread  the  practices  over  3  to  5  days. 
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5.10  Improve  job  aiding  and  train  students  extensively  on  using  TMs 
especially  where  multiple  TMs  are  required  to  perform  the  task. 

a.  Rationale:  The  Army  provides  and  mandates  the  use  of  TMs.  But, 
soldiers  use  them  very  little  or  not  effectively.  And,  even  if  they  do 
use  the  manuals,  they  still  get  hung  up  on  how  to  do  specific  6teps 
outlined  in  the  TMs  (Macpherson  et  al.,  1988a;  Kern  &  Hayes,  1983).  We 
think  a  combination  of  better  training  in  how  to  use  TMs  and  better  job 
aids  will  make  a  significant  dent  in  this  problem. 

b.  How-to: 

(1)  Increase  the  amount  and  quality  of  training  in  how  to  use  TMs, 
especially  where  multiple  TMs  are  required. 

(2)  Require  63W  students  in  Phases  1  and  2  to  use  manuals  in  all 
aspects  of  training.  Give  extra  experience  in  the  tasks  requiring 
multiple  manuals. 

(3)  Develop  job  aids  for  tasks  requiring  multiple  manuals  or  very 
difficult  searches  through  the  TMs,  e.g.  checklists  which  pull 
procedures  together  in  one  place.  Use  ARI's  Skill  Retention  Model  to 
identify  tasks  most  in  need  of  special  job  aiding. 

(4)  Example  of  Pay-off.  For  the  "engine  cranks  -  doesn't  start" 
problem,  the  50%  failure  rate  point  could  move  from  4  days  to  3.5  weeks 
with  improved  job  aiding. 

c.  For  tasks  with  8  or  more  facts  (e.g.  tolerances,  formulas, 
nomenclature)  do  the  following: 

(1)  Prepare  a  small,  well  indexed  card  catalog  with  the  most 
frequently  used  tolerances,  settings,  and  diagnostic  readings.  Again, 
the  Skill  Retention  Model  can  help  identify  what  set  of  facts  should  go 
in  the  catalog. 

(2)  Have  students  prepare  their  own  lists  of  facts  on  cards, 
protected  with  plastic.  Have  trainees  practice  reviewing  the  lists. 
Teach  the  facts  in  groups  of  3  or  4.  Do  this  before  they  practice  the 
whole  task,  and  then  after  they  finish.  For  facts  used  in  other  tasks, 
prepare  a  list  of  those  tasks  so  that  the  trainee  can  see  how  the  facts 
generalize.  Show  how  the  same  fact  (e.g  tolerance,  spec,  formula) 
applies  to  other  tasks.  Cross  reference  the  facts  by  page  number  to  the 
TM  or  TMs. 
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6.0  CONCLUSIONS 


6.1  Failure  among  wheel  vehicle  mechanics  to  perform  to  standard  is  a 
significant  but  predictable  problem  in  the  Army. 


6.2  Many  valid  techniques  and  strategies  exist  which  could  reduce  this 
condition  to  a  negligible  level. 


6.3  These  techniques  and  strategies  are  not  being  used,  or  are  being 
used  ineffectively 


6.4  To  implement  them  correctly,  wheel  vehicle  maintenance  training 
needs  to  shift  away  from  a  "stand-up  lecture/ content  oriented" 
philosophy  to  one  that  is  "workshop/task  oriented". 


6.5  The  techniques  and  strategies  are  fully  consistent  with  and  support 
TRADOC's  systems  approach  to  training  (SAT).  They  are  validly 
implemented  by  the  Harless  Workshop  on  Front-End  Analysis,  Job  Aiding, 
and  Course  Development. 


6.6  The  techniques  and  strategies  have  been  condensed  in  a  set  of 
guidelines  for  improving  63W10  training". 
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8.0  APPENDIX  1:  SUMMARY  OF  THE  HARLESS  METHOD 

FOR  DEVELOPING  TRAINING 


8.1  Front-End  Analysis  (FEA) : 

a.  Identify  each  specific  task  which  is  not  being  performed  to 
standard  on  the  job.  Then  decide  whether  the  performance  failure (s) 
results  from  lack  of  skill.  If  so  the  task  is  a  candidate  for  new  or 
improved  training. 

b.  Flow  chart  the  task  as  a  sequence  of  cues  and  actions  (stimuli  and 
responses) .  Show  where  these  cue-action  pairs  form  chains  of  behavior, 
discriminations  (i.e.,  decisions),  or  generalizations  (i.e.,  common 
action  for  different  situations) .  It's  critical  in  the  Harless  method  to 
identify  these  three  types  of  behavior,  because  Harless  provides  rules 
for  special  training  strategies  with  each  type. 

c.  Note  also  that  Harless  is  performance  not  content  oriented. 

Harless  agrees  with  the  ARI  guidebook  on  training  literature  (Kern  et 
al.,  1975)  that  you  should  teach  what  to  do  and  how  to  do  things. 


8.2  Training  Design. 

a.  Break  the  task  into  subtasks,  using  rules  provided  by  Harless,  and 
then  organize  the  subtasks  into  a  series  of  training  exercises  which 
Harless  calls  the  5-P  strategy:  Priming,  prompting,  practice,  isolated 
practice,  integrated  practice. 

b.  Priming  means  that  you  explain  or  demonstrate  a  subtask. 

Prompting  means  that  you  provide  self-paced  experiences  with  the 
subtask  but  with  lots  of  hints  about  how  to  perform  correctly.  Practice 
is  more  of  the  same,  but  with  less  prompting,  and  maybe  with  several 
subtasks  combined.  Isolated  practice  means  that  you  give  the  student 
experience  on  the  whole  task,  under  the  most  realistic  conditions 
possible.  Integrated  practice  means  that  you  combine  the  whole  task  with 
other,  related,  tasks. 

c.  The  5-P  strategy,  in  other  words,  is  just  a  well  organized  way  to 
move  the  student  from  part-  to  whole-task  training,  with  gradual 
increases  in  fidelity.  The  idea  certainly  is  not  new.  But  Harless  shows 
you  exactly  how  to  put  it  into  practice,  with  rules,  examples  and 
exercises.  In  other  words,  he  uses  the  5-P  strategy  to  teach  the  5-P 
strategy. 

d.  When  he  does  so,  he  uses  a  workshop  format,  i.e.,  a  live 
instructor  does  the  priming  and  then  turns  the  students  loose  on 
workbook  exercises.  But,  other  training  media  or  combinations  of  media 
can  be  used,  just  as  easily,  e.g.,  computer-aided  instruction  alone, 
or  in  combination  with  an  instructor. 
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terns.  Might  also  expand  the  defini¬ 
tion  of  generalization  by  setting  up 
tasks  designed  specifically  for  coping 
with  the  variety  of  conditions  under 
uhich  a  task  might  be  performed. 
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